Draft: Support Rails 8.1 + pure-Ruby PostgreSQL parity#1217
Draft: Support Rails 8.1 + pure-Ruby PostgreSQL parity#1217skunkworker wants to merge 5 commits into
Conversation
Target activerecord ~> 8.1.0 and bring the JDBC PostgreSQL adapter up to parity with the upstream pg adapter (pure-Ruby items only; DB warnings, which need Java SQLWarning plumbing, are deferred). Rails 8.1 compatibility: - log: accept the new allow_retry: keyword - check_version: raise below PostgreSQL 9.5 (was 9.3) - reintroduce check_if_write_query / mark_transaction_written_if_write (removed in 8.1) on top of write_query?/ensure_writes_are_allowed/ mark_transaction_written, so the adapter's custom query paths work - override decode_string_array / current_schemas to use the pure-Ruby array parser instead of PG::TextDecoder::Array (unavailable under JDBC), passing through arrays the JDBC driver already decodes Parity with upstream pg adapter: - add supports_exclusion_constraints?, supports_unique_constraints?, supports_deferrable_constraints?, supports_index_include? (PG 11), supports_restart_db_transaction? (PG 12), supports_close_prepared? (false) - add set_constraints and high_precision_current_timestamp - add exec_restart_db_transaction (JDBC-safe rollback + begin) - fix is_cached_plan_failure? to use JDBCError#sql_state (the PG::PG_DIAG_* path was dead under JDBC, so plan-cache-expiry retry never fired) - translate_exception: classify via SQLSTATE first (incl. CheckViolation and ExclusionViolation), keeping message matching as a fallback - update_typemap_for_default_timezone: drop dead PG::TextDecoder branch and reconfigure the connection when ActiveRecord.default_timezone changes test/simple.rb: Column#default now returns the cast-type-deserialized value in Rails 8.1, so assert typed defaults instead of raw strings. PostgreSQL suite: 283 tests, 0 failures, 0 errors (AR_VERSION=8.1.3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fix the connect_timeout property mapping and add knobs needed to run the JDBC PostgreSQL adapter safely behind a connection pooler. - connect_timeout now maps to pgjdbc connectTimeout (connection establishment), matching the pg gem / libpq meaning. It previously mapped to socketTimeout (a per-query read timeout). - add socket_timeout config -> pgjdbc socketTimeout, the per-query read timeout so a dead/reaped backend surfaces as an error instead of hanging the JVM thread. - add prepare_threshold config -> pgjdbc prepareThreshold passthrough so named server-side prepared statement promotion can be tuned/disabled without fully turning off prepared_statements (server-side named statements break under transaction/statement pooling). Applied to both the live config builder (adapter_hash_config.rb) and the legacy postgresql_connection path (connection_methods.rb). Document PostgreSQL connection/resilience options and add dedicated PgBouncer guidance (session vs transaction/statement pooling), covering prepared_statements, advisory_locks, and per-session SET state. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@headius opus also picked up on a discrepancy in the The
|
What it should be (connect_timeout) |
What it was wired to (socketTimeout) |
|
|---|---|---|
| Phase governed | TCP connection establishment | Every read on an already-open connection |
| Fires when | The server can't be reached in time | A single query takes too long to return data |
| pgjdbc equivalent | connectTimeout |
socketTimeout |
Concrete consequences of the old mapping:
- Setting
connect_timeout: 30silently imposed a 30-second cap on every
query's read time — a long-running report query would be killed at 30s,
surprising anyone who set it expecting only to bound connection setup. - Meanwhile actual connection establishment kept pgjdbc's own
connectTimeout
default (10s) and was never influenced by the user's setting. - There was no way at all to set a real per-query read timeout
(socketTimeout), which is exactly the resilience knob you want behind
PgBouncer.
The reference behavior (with sources)
libpq / pg gem — connect_timeout is connection establishment:
"Maximum time to wait while connecting, in seconds... Zero, negative, or not
specified means wait indefinitely."— PostgreSQL libpq docs, Connection Parameters:
https://www.postgresql.org/docs/current/libpq-connect.html
(The pg gem passes connection options straight through to libpq, so its
connect_timeout carries this exact meaning — this is the behavior we align
with.)
pgjdbc — the two distinct properties:
connectTimeout— "The timeout value used for socket connect operations. If
connecting to the server takes longer than this value, the connection is
broken... a value of zero means that it is disabled." Default:10seconds.
socketTimeout— "The timeout value used for socket read operations... This
can be used as both a brute force global query timeout and a method of detecting
network problems... a value of zero means that it is disabled." Default:0
(disabled).— pgjdbc docs, Connection Parameters:
https://jdbc.postgresql.org/documentation/use/#connection-parameters
So the correct mapping is connect_timeout → connectTimeout (matching libpq/pg),
with socketTimeout exposed separately.
The fix (commit 90eeb464, branch v81)
# connect_timeout -> connectTimeout (matches pg gem / libpq)
properties["connectTimeout"] ||= connect_timeout if connect_timeout
# new: socket_timeout -> socketTimeout (per-query read timeout)
properties["socketTimeout"] ||= socket_timeout if socket_timeoutThis makes connect_timeout behave as users coming from the pg gem expect, and
introduces socket_timeout for the read-timeout role the old code was conflating
it with. The behavior change is called out in the README's "Connection and
resilience options" note for anyone who relied on the old side effect.
Relocate the PostgreSQL connection/resilience options and PgBouncer guidance from the top-level README into activerecord-jdbcpostgresql-adapter/README, leaving a short pointer in the top-level README to keep it focused. Convert that sub-gem README from .txt to .md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bring the JDBC PostgreSQL adapter to Rails 8.1 feature parity for two result-side features the pg adapter exposes. DB query warnings (config.active_record.db_warnings_action): - Java: RubyJdbcConnection collects java.sql.SQLWarning after each execute into a per-connection buffer, exposed to Ruby via #take_warnings. Gated by #collectsWarnings (PostgreSQL-only) so other adapters pay no cost. PostgreSQLRubyJdbcConnection extracts the server severity (WARNING/NOTICE) from PSQLWarning for level-based filtering. - Ruby: ArJdbc::PostgreSQL#handle_warnings drains the buffer and dispatches via ActiveRecord.db_warnings_action, honoring #warning_ignored? (db_warnings_ignore + severity). Wired into the JDBC query paths, which bypass the abstract adapter's #raw_execute-based handling. Always drains so the buffer can't grow even when warnings are ignored. affected_rows on ActiveRecord::Result: - PostgreSQLResult#toARResult sets @affected_rows (row count) on the result, and #cmd_tuples/#cmdtuples now returns the row count instead of a bogus first-cell value. Negative tests: - exception_translation_test: SQLSTATE -> ActiveRecord error mapping (unique/FK/not-null/check/exclusion/value-too-long/range/statement-invalid), prepared + non-prepared, asserting JDBCError#sql_state. - db_warnings_test: warning captured/raised, and ignored by level (NOTICE), by db_warnings_ignore, and when absent. - version_test: stub the new take_warnings on the mocked raw connection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lib/arjdbc/postgresql/connection_methods.rb is dead code: its require is commented out in lib/arjdbc/postgresql.rb and nothing else loads it. The live connection-property logic is in adapter_hash_config.rb, so the earlier pgbouncer edits here were a misleading duplicate. Revert this file to match master and keep the change in one place. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
headius
left a comment
There was a problem hiding this comment.
I believe Claude needs to take a look at the comments I've provided, with the additional context that this project relies on the standard Rails ActiveRecord tests for verification. The additional goals or prompts in simple terms would be something like:
activerecord-jdbc-adapter is intended to be a minimal set of diffs based on rails/rails activerecord adapters, only as needed to replace the native drivers used by rails/rails with JDBC equivalents. As such, we should always try to reuse rather than copy utility code, we should try to mimic the evolving structure of utility code in each adapter, and any diff from the original logic should include justification for why the Rails equivalent cannot be reused.
Additional finer-grained goals for any refactoring or updating of this codebase:
- Avoid duplicating logic from Rails if it can be called from ARJDBC.
- Avoid copying tests from Rails if there's no significant change that's specific to ARJDBC. Assume the Rails copy of the tests will be used to verify behavior along with our own tests.
- Wherever possible, match the Rails structure of DB utility methods rather than duplicating such methods' content multiple places in our codebase. This is a compatibility project that aims to have as little custom code as possible so introducing more copied code makes that goal harder.
- Provide correlating comments when changes must be made to ARJDBC that are equivalent to, but uncopyable from, the Rails version of the same code. Link to specific lines or ranges of lines and provide justification for copying the change versus calling the original code.
- Where possible, additional work or research could be done to recommend changes to rails/rails that improve the reusability of code. This would include things like moving utility code common to both the native and JDBC adapters into modules or making stateless utility code into class-level utility functions.
Overall the changes I reviewed seem logical and I was able to correlate most of them with equivalent lines in rails/rails. The tests are too big of a copy and Claude needs to revisit those with the consideration that we also run them from rails/rails.
| @@ -0,0 +1,114 @@ | |||
| # activerecord-jdbcpostgresql-adapter | |||
There was a problem hiding this comment.
Is this content generated? I cannot verify all of this without knowing where it came from.
|
|
||
| # this version of log() automatically fills type_casted_binds from binds if necessary | ||
| def log(sql, name = "SQL", binds = [], type_casted_binds = [], async: false, &block) | ||
| def log(sql, name = "SQL", binds = [], type_casted_binds = [], async: false, allow_retry: false, &block) |
There was a problem hiding this comment.
| end | ||
| end | ||
| end | ||
| handle_warnings(sql) |
There was a problem hiding this comment.
Roughly equivalent to the following section, but Rails moves some of this boilerplate into a perform_query method we probably want to mimic.
Also, this change and several below are a single method in Rails: raw_execute. Duplicating this change everywhere is perhaps fine for now but we need to move toward matching the structure of utility code in Rails. Ideally, we would work with Rails Core to make more of this code reusable.
| check_if_write_query(sql) if respond_to?(:check_if_write_query, true) | ||
| mark_transaction_written_if_write(sql) if respond_to?(:mark_transaction_written_if_write, true) | ||
| sql | ||
| # AR >= 8.1 removed these helpers, folding them into #preprocess_query via |
There was a problem hiding this comment.
Fine for now but we are introducing more divergence from the original.
| execute("SET time zone '#{tz}'", 'SCHEMA') | ||
| end unless redshift? | ||
|
|
||
| # Track the timezone the session was configured for so #update_typemap_for_default_timezone |
There was a problem hiding this comment.
This appears to be a way to handle the differing timezone handling in the JDBC driver versus the PG gem but I have to wonder if there's a cleaner way.
| connect_timeout = config[:connect_timeout] || ENV["PGCONNECT_TIMEOUT"] | ||
|
|
||
| properties["socketTimeout"] ||= connect_timeout if connect_timeout | ||
| properties["connectTimeout"] ||= connect_timeout if connect_timeout |
There was a problem hiding this comment.
This is the extra fix for "connect_timeout discrepancy" mentioned in the PR and appears to be valid based on PG documentation here:
https://jdbc.postgresql.org/documentation/use/
connectTimeout (int) Default 10
The timeout value used for socket connect operations. If connecting to the server takes longer than this value, the connection is broken. The timeout is specified in seconds max(2147484) and a value of zero means that it is disabled.socketTimeout (int) Default 0
The timeout value used for socket read operations. If reading from the server takes longer than this value, the connection is closed. This can be used as both a brute force global query timeout and a method of detecting network problems. The timeout is specified in seconds max(2147484) and a value of zero means that it is disabled.
| properties["prepareThreshold"] = 0 | ||
| end | ||
|
|
||
| # :prepare_threshold - explicit pgjdbc prepareThreshold passthrough. Lets |
There was a problem hiding this comment.
From https://jdbc.postgresql.org/documentation/use/:
- prepareThreshold (int) Default 5
Determine the number of PreparedStatement executions required before switching over to use server side prepared statements. The default is five, meaning start using server side prepared statements on the fifth execution of the same PreparedStatement object. A value of -1 activates server side prepared statements and forces binary transfer for enabled types (see binaryTransfer ). More information on server side prepared statements is available in the section called Server Prepared Statements.
| # JDBC driver returns timestamps based on the session time zone set in | ||
| # #configure_connection. So if ActiveRecord.default_timezone changes at | ||
| # runtime we just re-run #configure_connection to apply the new zone. | ||
| if @default_timezone != ActiveRecord.default_timezone |
There was a problem hiding this comment.
See comments about handling timezone discrepancies above.
| statement.executeUpdate(query, createStatementPk(pk)); | ||
| } | ||
|
|
||
| collectWarnings(context, statement); |
There was a problem hiding this comment.
Seems like a lot of duplication here and I am unsure what the equivalent logic looks like in Rails.
| @@ -0,0 +1,87 @@ | |||
| require 'db/postgres' | |||
There was a problem hiding this comment.
This is where tests start, and as mentioned in #1218 there's definitely going to be a lot of duplication and copied code here.
Target activerecord ~> 8.1.0 and bring the JDBC PostgreSQL adapter up to parity with the upstream pg adapter (pure-Ruby items only; DB warnings, which need Java SQLWarning plumbing, are deferred).
Rails 8.1 compatibility:
Parity with upstream pg adapter:
test/simple.rb: Column#default now returns the cast-type-deserialized value in Rails 8.1, so assert typed defaults instead of raw strings.
PostgreSQL suite: 283 tests, 0 failures, 0 errors (AR_VERSION=8.1.3).